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Abstract. We classify extremal curves in free nilpotent Lie groups. The classifi- 
cation is obtained via an explicit integration of the adjoint equation in Pontryagin 
Maximum Principle. It turns out that abnormal extremals are precisely the hori- 
zontal curves contained in algebraic varieties of a specific type. We also extend the 
1 results to the nonfree case. 
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1. Introduction 



Let M be a differentiable manifold and T> C TM a bracket generating distribution. 
A Lipschitz curve 7 : [0, 1] — > M is horizontal if ^(t) G T>(^y(t)) for a.e. t e [0,1]. 
Fixing a quadratic form on D, one can define the length of horizontal curves. A 

O: 

distance on M can be defined by minimizing the length of horizontal curves connecting 
pair of points. This is known as sub-Riemannian or Carnot-Caratheodory distance. 
If the resulting metric space is proper, length minimizing curves between any given 
pair of points do exist. 

The main open problem in the field is the regularity of length minimizing curves, 
see Problem 10.1]. Length minimizing curves may in fact be abnormal extremals in 
the sense of Geometric Control Theory: while normal extremals are always smooth, 
abnormal ones are apriori only Lipschitz continuous. We refer the reader to Section [2] 
for precise definitions. Let us only recall here that a horizontal curve is an abnormal 
extremal if the end-point mapping is singular at this curve, i.e., its differential is not 
surjective. The notion of abnormal extremal is also related to that of rigid curve, 
I see [3] 114]. Abnormal extremals depend only on the structure (M, T>) and not on the 

fixed quadratic form on T>; it is well known that they can be length minimizing, see 
[TO] [12] . On the other hand, in some structures the presence of singularities prevent 
^ ■ an abnormal extremal to be length minimizing [HI [IT] . To our best knowledge, no 

example of minimizing nonsmooth horizontal curve is presently known. 

The set of abnormal extremals of a structure (M, T>) is an interesting object that 
is not yet well- understood. See, however, the deep second order analysis [2j. In this 
paper, we describe this set when M is a (connected, simply connected) free nilpotent 
Lie group and T> is the left-invariant subbundle spanned by a system of generators of 
its Lie algebra. We obtain a description that is constructive and of a purely algebraic 
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type. It is constructive in the sense that it permits to construct abnormal extremals of 
any (feasible) desired type: Goh extremals, extremals with given corank, extremals of 
minimal order in the sense of [I] , etc. The classification is purely algebraic in the sense 
that it only depends on the structure constants of the algebra of the group. These 
results also extend to other nilpotent groups. In particular, we consider connected, 
simply connected, nilpotent and stratified Lie groups (Carnot groups): this is of 
special interest because, by Mitchell's theorem, Carnot groups are the infinitesimal 
models of equiregular sub-Riemannian structures. 

Let us give some flavour of the results contained in the paper. We start by fixing 
a basis X\,...,X n of an n-dimensional free nilpotent Lie algebra $j generated by 
r elements X\, . . . ,X r ; the choice of this basis has to be done according to a very 
precise algorithm due to M. Hall Jr., see [TJ. A detailed description of this algorithm 
is contained in Section [3J This basis determines a collection of generalized structure 
constants, see (14. 42 p . Using these constants, for any % = 1, . . . ,n and for any multi- 
index a G X := N n = {0, 1, 2, . . . } n , we define certain linear mappings cj>i a : W 1 — > M, 
see f )4.43p . When |a| :=«! + • • -+a n is large we have 4> ia = because of the nilpotency. 
For each % — 1, . . . , n and v G M n , we introduce the polynomials P? : M. n — > R 



where we let ° n . These polynomials enjoy some remarkable properties 

that are discussed in Proposition 14.51 

A result due to M. Grayson and R. Grossman [6] ensures that g is isomorphic to 
a certain algebra of vector fields in M n ; it turns out that G can be identified with 
R n via exponential coordinates of the second type associated with the vector fields 
corresponding to X\, . . . , X n , see Proposition 13.51 For any v G M. n , we call the set 



an abnormal variety of G of corank 1; when u^Owe have Z v ^ K n , see Proposition 

One of the results proved in the paper is the following theorem; see Definition 12.31 
for the notion of corank. 

Theorem 1.1. Let G = W a be a free nilpotent Lie group and let 7 : [0, 1] — > G be a 

horizontal curve with 7(0) = 0. The following statements are equivalent: 

A) The curve 7 is an abnormal extremal of corank m > 1 . 

B) There exist m linearly independent vectors V\, . . . ,v m G lR n such that ^(t) G 



A stronger version of Theorem 11.11 holds when 7 is a strictly abnormal extremal. In 
this case, the Goh condition (see Theorem 12.41) implies that for all t G [0, 1] we have 



(1.1) 




Z v = {x G R n : P?(x) 



Z vi n...H Z Vm for all t G [0, 1] . 



(1.2) 



7(t) e{xeR n : P?(x) 



p:(x) = o}, 
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where s = dim(gi © [fli,0i]) and Qi is the first layer of g, i.e., the subspace spanned 
by the generators of g. We call the algebraic sets as in ( 11. 2p Goh varieties of G. 

The proof of Theorem 1 1 . 1 1 relies upon Theorem 14. 6 \ where, for any extremal (normal 
or abnormal), we explicitly compute the dual curve provided by Pontryagin Maximum 
Principle. We started from the usual version of the differential equation for the dual 
curve in Optimal Control Theory, see Theorem 12. 1| formula (12. 6p . The coordinates 
A = (Ai, . . . , A n ) of the dual curve in the basis of 1-forms dual to the fixed frame of 
vector fields, see (I2.15p . satisfy the system of equations (I2.17p . After studying some 
Lie groups of step 4, 5 and 6, we could guess the general formula for A. This led us 
to the polynomials in ( II. lft : indeed, in Theorem I4.6| we prove that 

(1.3) Xi(t) = P?{-y{t)), for all t e [0, 1] and i = 1, . . . , n, 

where v := A(0) ^ is the initial condition. To check formulae (II. 3p . we had to 
understand how the Jacobi identity enters the process of integration of the adjoint 
equation. The key technical tool is a combinatorial identity for iterated commutators 
in free nilpotent Lie algebras that is proved by induction in Lemma 13.21 We were not 
able to find it in the Lie algebra literature. 

Formulae (ll.3p provide an explicit integration of the second equation in the Hamil- 
tonian system (I2.9p . see Remark 14.71 These formulae hold for both normal and 
abnormal extremals and, from the technical viewpoint, are the main result of the 
paper. 

In Section |5l we extend our results from the free case to the case of nonfree Carnot 
groups. The results here are less explicit because they involve a "lifting" proce- 
dure from a nonfree group to a free one, see Theorem 15.41 However, the results are 
precise enough to deduce in a purely algebraic way some interesting facts on Carnot- 
Caratheodory geodesies, such as the C°° smoothness of length minimizing curves in 
Carnot groups of step 3 (see [13]), and Gole-Karidi's example |5J of a strictly abnor- 
mal extremal in a Carnot group. These and other examples are briefly discussed in 
Section EJ 

The results of this paper raise several questions. As noticed, abnormal extremals 
are precisely horizontal curves lying inside abnormal varieties: consequently, the in- 
terplay between the horizontal distribution and the tangent space to the variety de- 
termines the structure of all possible singularities of abnormal extremals. The precise 
knowledge of these singularities could permit a fine analysis of the length minimality 
properties of abnormal extremals. It would be interesting to use the techniques de- 
veloped in [5] and [TT] to exclude corners and other kinds of singularities for length 
minimizers. See, however, the nonsmooth extremal of Section 16.41 

Also, it would be interesting to understand whether our results can be extended 
to wider classes of Lie groups or to other families of sub-Riemannian structures. For 
instance, one could wonder whether abnormal extremals of analytic sub-Riemannian 
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manifolds (namely, analytic manifolds with analytic horizontal distribution) are con- 
tained in an analytic variety and whether such varieties can be characterized in some 
way. 

As noted above, the family of abnormal extremals is a geometric object associated 
with a manifold M along with a bracket generating distribution D C TM. In the 
same spirit, an abnormal variety is an intrinsic algebraic subset of a (free) Lie group. 
The study of abnormal varieties could be of interest in real algebraic geometry. Na- 
tural questions concern generic dimension, smoothness, structure of singularities and 
topology of abnormal varieties. In Section I6.4[ we list the quadrics defining Goh 
extremals in the free nilpotent Lie group of rank 3 and step 4. 

2. Extremal curves in sub-Riemannian geometry 

In this section, we recall some basic facts concerning extremal curves in sub- 
Riemannian geometry. Let X±, . . . ,X r , r > 2, be linearly independent smooth 
vector fields in R n , n > 3, and let T> denote the distribution of r-planes T>(x) = 
span{Xi(x), . . . ,X r (x)} with x £ W 1 . V is called horizontal distribution and its sec- 
tions are called horizontal vector fields. With respect to the standard basis . . . , 
of W 1 we have for any j — 1, . . . , r 

n 

i=l 1 

where Xji : M. n — > K are smooth functions. 

A Lipschitz curve 7 : [0, 1] — > R n is V- horizontal, or simply horizontal, if there 
exists a vector of functions h = (hi, . . . , h r ) £ L°°([0, 1]; M r ) such that 

r 

i=i 

The functions h are called controls of 7. Let g x be the quadratic form on T>(x) making 
Xi, . . . , X r orthonormal. The horizontal length of a horizontal curve 7 is then 

/* P X/2 

m = ( j o gyitMwt) = ( y o \h{t)\ 2 dt) . 

Here, we are adopting the L 2 definition for the length. For any couple of points 
x,y £ R n , we can define the distance 

(2.5) d(x,y) = inf |-^(7) : 7 is horizontal, 7(0) = x and 7(1) = yj. 

If the above set is nonempty for any x,y £ W 1 , then d is a distance on W 1 , usually 
called Carnot-Caratheodory distance. This holds when the vector fields X\, . . . ,X r 
satisfy the Hormander bracket-generating condition: at any point of M n , X\, . . . ,X r 
along with their commutators of sufficiently large length span a vector space of full 
dimension n. 

If the resulting metric space {W 1 , d) is complete, the infimum in ( 12. 5 p is attained. 
We call a curve 7 providing the minimum a (length) minimizer. If h is the vector of 
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the controls of a minimizer 7, we call the pair (7, h) an optimal pair. The minimizer, 
which in general is not unique, is found within the class of Lipschitz curves. Pontrya- 
gin Maximum Principle provides necessary conditions for a horizontal curve to be a 
minimizer. 

Theorem 2.1. Let (7, h) be an optimal pair. Then there exist £ £ {0,1} and a 
Lipschitz curve £ : [0, 1] — > R n such that: 

i)£o + |£l t^O on [0,1]/ 

h) Zohj + (£, Xj(i)) = on [0, 1] for all 3 = 1, • • • , r; 

iii) tne coordinates k = 1, . . . , n ; 0/ i/ie curi> e £ so/ue tne system of differential 
equations 

r n dX 

(2.6) ^"EE-^W^' a ' e - 

j=i j=i fc 

We refer to [T[ Chapter 12] for a proof of Theorem 12.11 in a more general framework. 
Equations (12.61) are called adjoint equations. 

Definition 2.2. We say that a horizontal curve 7 : [0, 1] — y R n is an extremal if 
there exist £0 £ {0, 1} and £ G Lip([0, 1]; R n ) such that i), ii), and iii) in Theorem \2.1\ 
hold. 

We say that 7 is a normal extremal if there exists such a pair (£ , £) with ^ 7^ 0. 
VFe say that 7 zs an abnormal extremal if there exists such a pair with £ = 0. 
We say that 7 is a strictly abnormal extremal i/7 is an abnormal extremal but not 
a normal one. 

We call a curve £ satisfying i), ii), and iii) in Theorem 12.11 a dual curve of 7. We 
identify the curve £ with the curve of 1-forms in IR n 

f = fjdarj + . . . + £ n da; n . 

The dual curve £ is constructed in the following way. Let $ : [0, 1] x R™ — y R" be 
the flow in R n associated with the controls hi,...,h r G L 2 ([0, 1]) of 7. Namely, let 
$(t, x) = 7a;(t) where 7^ : [0, 1] — y R" is the solution to the problem 

r 

7x = Yl h J X .i(" ' ■ a - e - and t»(o) = ^ e Rrt - 

We also let $ t (a;) = §(t,x). Then, dual curves £ : [0, 1] — y T*R n are of the form 

(2.7) £(f) = ($r 1 )*e(0), tG[0,l] 

for suitable £(0) 7^ 0. Above, (^>^~ 1 )*^'(0) denotes the pull-back of 1-forms by the 
diffeomorphism $7/ x : R n — > R n - We refer to pQ for this characterization of dual 
curves. 

Normal extremals are smooth curves. In fact, by ii) the controls satisfy 

(2.8) ^ = -(£,X,( 7 )) a .e. on [0, 1] 
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for any j = l,...,r. This along with the adjoint equation (12.61) implies the C°°- 
smoothness of 7. Moreover, the pair (7, £) solves the system of Hamilton's equations 

(2.9) 7 = ^(7,f), f=-^(7,0, 
where H is the Hamiltonian function 

For abnormal extremals we have, for any j — 1, . . . , r, 

(2.10) (£,X,-( 7 )) = on [0,1]. 

Let us recall the definition of the end-point mapping with initial point i 6 1". For 
any h G L 2 ([0, 1]; W), let 7 h be the solution of the problem 

r 

7 h = ^^X J ( 7 h ), 7 h (0) = * . 
i=i 

The mapping £ : L 2 ([0, l];R r ) — > M n , £(h) = 7 (1), is called the end-point mapping 
with initial point Xo- It is well known that a horizontal curve 7 starting from x$ with 
controls ft, is an abnormal extremal if and only if there exists A G M n , A ^ 0, such 
that 

(2.11) (d£(h)v,X) = 

for all v G £ 2 ([0, 1]; IR r ). Here, d£(h) is the differential of £ at the point h. Abnormal 
extremals are precisely the singular points of the end-point mapping. Let Im d£(h) C 
W 1 denote the image of the differential d£(h) : L 2 ([0, 1]; R r ) ->■ R n . 

Definition 2.3. T7ie corank 0/ an abnormal extremal 7 : [0, 1] — > IR n with controls h 
is the integer n — dim(lmdf (h)) > 1. 

If 7 has corank m > 1 then we have m linearly independent functions £ 1 , . . . G 
Lip([0, l];M n ) each solving the system of adjoint equations (12. 6J) . 

The necessary condition (12. lOj) can be improved in the case of strictly abnormal 
minimizers. 

Theorem 2.4. Let 7 : [0, 1] — > M" fre a strictly abnormal length minimizer. Then 
any dual curve £ G Lip([0, l],M n ) satisfies 

(2.12) (e,[X i ,X,]( 7 )) = on [0,1] 
for any i,j = 1, . . . , r. 

Condition (12.121) is known as Goh condition. Theorem 12.41 can be deduced from second 
order open mapping theorems. We refer to [1, Chapter 20] for a systematic treatment 
of the subject. 
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Definition 2.5. We say that a horizontal curve 7 : [0, 1] — > IR n is a Goh extremal 
if there exists a Lipschitz curve £ : [0, 1] — > MP such that £ 7^ 0, £ solves the adjoint 
equations (12. 6p and (£,^(7)) = (£, [Xj, Xj](j)) = on [0, 1] /or all i,j = 1, . . . , r. 

Goal of this paper is to integrate the system of adjoint equations. We shall actually 
integrate an equivalent system of ordinary differential equations. To this aim, let us 
complete the system of vector fields X\, . . . , X r to a frame of n linearly independent 
vector fields X\, . . . ,X n . This is always possible locally. Then there are smooth 
functions c& such that 

n 

(2.13) [X u X J } = Y,4X k , i.j 1 n. 

k=l 

In Lie groups, if X\, . . . , X n form a basis of left invariant vector fields, the functions 
c\j are constants called structure constants of the group. 

Let . . . , $ n be the frame of 1-forms dual to the frame of vector fields X\, . . . , X n . 
In the standard basis dx\, . . . , dx n , we have for suitable functions : M. n — > M. 

n 

k=i 

Then, for alH, j = 1, . . . , n, we have 

n 

(2.14) 5 ij = & i (X j ) = y %2& ik X jk . 

k=l 

The coefficients are defined as in f)2.4p . for all j = 1, . . . ,n. Here and hereafter, 
5ij is the Kronecker symbol. 

Given an extremal curve 7 with dual curve £, let Ai, . . . , A„ € Lip([0, 1]) be the 
functions defined via the relation along 7 

(2.15) frdxi + ■■■+ indx n = Ai0i(7) + ■ ■ • + A„^ n (7). 

We translate the adjoint equations (I2.6P for £ into a system of differential equations 
for the coordinates Ai, . . . , A n of £ in the frame . . . , i? n - We can express 7 in the 
standard coordinates of M. n as 7 = (71, . . . , j n ). In the case of Lie groups, the following 
theorem is proved in [5]. 

Theorem 2.6. Assume that the vector fields X x , . . . ,X r satisfy 

(2.16) X jh = 6 jh for all I <j,h<r. 

Then the functions £1, ... ,£ n solve equations f 12 . 6 j) if and only ij 'the functions Ai, ... , A r 
satisfy the system of differential equations 

n r 

(2.17) Ai = - J2J2 C U^ X " a - e - on t ' 1 ]' 

fc=i 3=1 

for any i = 1, . . . , n. 
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Proof. Let h : [0, 1] —> W be the controls of 7: 

7 = Mfi(7) + • • • + M^r(7) a.e. on [0, 1]. 

By (I2.16P we have jj = hj a.e. on [0, 1], j = 1, . . . , r. On the one hand, by differenti- 
ating the identity 



6c = £^0^(7), 



we obtain 



it 

ik = £^ + ^^(7), 7) 
i=i 

= £ ( ^ + ^( v ^(t), £ h 3 x Ai)) ) 

8=1 ^ ' j=l ' 

= £ (wo* + A * £ h i x Ak(i)) ■ 
i=i ^ 7=1 / 



On the other hand, equation (I2.6P reads 

r n ny. n 

4 = -££^(7)£aa( 7 ). 

j=1 i=l fc h=l 

Taking the difference of the previous two identities, we get 



n n r r n n a v 

= £ \A k + £ £ Aj/ij x^ fc ( 7 ) + £ £ £ ^(^AW, 

i=l i=l j=l j=l i=l ft=l 

for any k — 1, . . . , n. It follows that for any p = 1, . . . , n we have 

n / n n r r n PiV 

= £ x pk £ A^ fe + £ £ a^ x^ lfc ( 7 ) + £ £ fy-^%)A,A<(7) 

k=l \i=l i=l j=l j=l i,h=l k 

Notice now that, by fl2TT3j) . 

n n 

(2.18) = £x pfc (x^ fc ) + £(x i x pfe )A fe ; 

k=l k=l 

moreover, 

(2-19) £^^ 2i (7) = (^,)( 7 ). 

fc=i fc 
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Using (JZJHD, ( EEHD , and (12~T9]) . we obtain 

?i r r n 

j,fc=l J=l J=l i,/i=l 

n r r n 

=vEE A ^ ( x ^)(7) ^(7) + HH w*p*7k)(7) ^(7) 

n r 

= A P + ^ h flik{l) (X p X jk - XjXpfc) (7). 

j,A;=i i=i 
From (I2.13P we deduce that 

n 

(X p X jk - XjX^i) = ^ cpj ( 7 )x ft ( 7 ), 

e=i 

and hence, by (12.141) . we have 

n r n r 

= A P + ^^^(7)^(7)^(7) = A P + ^^A,V;,(7)- 

This is equation (I2.17p . The same computations show that (12.61) follows from (I2.17p . 

□ 

3. Hall basis theorem for free nilpotent Lie algebras 

In this section, we develop the algebraic tools needed in Section @] to integrate the 
system of differential equations (12.171) in free nilpotent Lie groups. In groups, the 
structure functions cf- appearing in (12.131) and (12.171) are in fact constants, as soon 
as the vector fields Xi, . . . ,X n are left invariant. The technical difficulty is to make 
transparent how the Jacobi identity enters the integration process. The algebraic 
preliminaries for this integration are fixed in Lemma 13.21 

Let q be a free nilpotent real Lie algebra of dimension n, step s > 2 and rank r > 2. 
The algebra q admits a stratification 

= 01©---©0 S , 

where = [0i, 0i] for % = 1, . . . , s — 1 and Qi = {0} for i > s. The first layer 0! has 
dimension r. 

We recall the algorithm for the construction of a basis for g due to M. Hall Jr. [7j. 
Let X±, . . . , X r be a basis for To each Xi we assign the degree d{i) = deg(Xj) = 1 
for i = 1, . . . , r. By induction, we complete X±, . . . , X r , to a basis X±, . . . , X n for q, 
and we assign to each Xj a degree d(j) = deg(Xj) G {1, . . . , s}. Let us assume that 
the elements Xi, . . . X k , with degree at most d — 1, are already defined and that they 
are ordered with the property: 

d(i) < d — 1, d(j) < d — 1, and d{i) < d(j) =>- i < j. 
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By definition, the commutator [Xj,Xj] is an element of the Hall basis of degree d if: 
(3.20a) i > j, 

(3.20b) Xi and Xj are elements of the basis of degree < d — 1, 

(3.20c) d(i) + d{j) = d, 

(3.20d) if Xi was constructed as [Xh, X&], then k < j. 

Hall proved that this algorithm produces a basis X 1; . . . , X n for q. The basis is ordered 
by subindices in such a way that d(i) < d(j) implies % < j. 

Following the genesis of the basis, it is easy to see by induction on the degree that, 
for any £ = 1, . . . , n there exists a unique string of indices £ , £\, £ 2 , ■ ■ ■ , £h £ {!)•••> n }, 
with h G {0, . . . , s — 1}, such that 

(3-21) X e =[.--[[[X t0 ,X tl ],X h ],...),X th \ 
and 

(3.22a) £ > £ u 

(3.22b) d(£ ) = = 1, 

(3.22c) £ x < £ 2 < . . . < £ h , 

(3.22d) iiX k =[---[[[X io ,X il },X i2 },...},X ii _ 1 ], withi = 2,...,/i, then £ { < k. 

We need some more notation. Let us first introduce the set of multi- indices X = N n . 
For any a G X, we let \a\ — a± + . . . + a n and u a = m&x{i : «j ^ 0}. Let us agree 
that a < (3 for a, (3 G X means a« < for all % = 1, . . . , n. 

Fix a Hall basis X±, . . . , X n . For j , ji, j 2 , • • • , jk £ {1, • • • , n}, we let 

(3.23) [X jo , X h , X j2 ,X j3 ,..., X jk ] := [■ ■ ■ [[[X jo , X h ], X j2 ] , X j3 ] , . . . , XJ . 
For a = (ai, . . . , a„) G X, we define the iterated commutator 

(3.24) [;X a ] := t-, ^, X^Xa,. Xa , . . . 

ai times Q2 times 

We agree that [•, X( ,...,o)] = Id. 

For any £ G {1, . . . , n) there exist unique £q G {2, . . . , r} and h = h(£) G {1, . . . , s — 
1} such that Xg is given by the representation (I3.2ip subject to (I3.22ap -( l3.22dj) . For 
each £ we define the multi-index I(£) G X as 

(3.25) I(£) j = #{ s >l:£ s =j}. 

For example, if X e = [X 3 ,X 2 ], then 4 = 3 and I(£) = (0, 1, 0, . . . , 0). Also, if 
£ G {1, . . . , r} then £ = £ and I{£) = (0, . . . , 0). 

With this notation, we have the following properties, for any £ — 1, . . . , n: 

(3.26) Xg = [X^ ,X/(£)], 

(3.27) m /w = £ m . 
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Moreover, given £, k G {1, . . . , n}, we have 

(3.28) [X^Xfc] is a base vector -<=>• uj^ < k < £. 

Definition 3.1. We say that Xg = [Xg , X^, JQ 2 , . . . , XgJ is a direct discendant of 
Xj if Xj = [Xi , X4 , Xi 2 , . . . , Xe k ], for some k G {0, . . . , h}. In this case, we write 
I <(■ 

The relation ^ is a partial order on indices. Notice that j ' ^ £ implies I(j) < I{£), 
but not viceversa. 

The next combinatorial lemma plays a central role in the next section. For any 
(3 G X and q = 1, . . . , r, let us define the set of indices 

Af,, q = {ee{i,...,n}--q±t, m<P}- 

The statement q ^ £ is equivalent to £ = q. We will also use the notation [3\ : = 
ft! |0| := ft + ••• + ft, and e k := (6 lk , 5 2k , S nk ) = (0, . . . , 1, . . . , 0). 

Lemma 3.2. For all G X, i = 1, n, and q — 1, r, we have 

(3-29) [[Xi, X g ], Xp] = c ep[Xi, Xp_j(£) +ei ,], 

where we let 

0! 

(3 - 30) ^ " (0 -/(£))!/(£)!■ 

Proof. The case /3 = is straightforward. The proof is by induction on |/3| > 1. 

.Base o/ induction. Assume \(3\ = 1, i.e., (3 = e k , for some k G {1, 2, ... , n}. Hence, 
/3! = 1. Let £ G Since I(£) < (3, then either I{£) = (0, . . . , 0) or I{£) = e k . 

In both cases we have c#p = 1. Since £o — <2S then Xi is either X,j, in which case 
f3 — I(£) + eg = e k + e g , or X^ = [X q , X k ], in which case (3 — I{£) + e^ = e^. Notice 
that the latter case is allowed only if k < q. 

Therefore, equation (13.291) reduces to a trivial identity when k > q, whereas when 
k < q it reduces to 

[[Xj, X q ], X k ] = [[Xi, X k ], X q ] + [Xi, [X q ,X k ]], 

which is true by the Jacobi identity. 

Inductive step. Using induction, we prove the claim for any j3 G X. We write 
f3 = (3 + e k for some f3 G X \ {0} and k = u-s G {1,2,..., n}. Hence we have Up < k 
and |/3 1 < \(3\. By induction, formula (13.291) holds for (3 and thus: 

[[Xj, X g ], X^] = 2^ C ?l3[[Xi, X J g_/(£) +e J, Xfc]. 

We split the sum into two sums according to whether £ < k or £ > k. In the case 
when £ < k, the coordinates of the multi-index (3 — I{£) + from the (k + l)-th 
onward are null and hence, 

[[Xj,X jS „/ W+e J,X fc ] = [Xi,Xp_!^ +ee+ek ]. 
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In the case I > k, also because Ui^\ < k, the commutator [X^,Xfc] is an element of 
the Hall basis, see (I3.28p . We denote this element by Xtk = [Xi,Xk]- Then by the 
Jacobi identity we obtain 

[[Xi, Xp-i(e) +Ce ], Xk] = [[[Xi,Xp-i(g)],Xe],Xk] 

= [[[Xi, Xp_i(£)\, Xk], X e ] + [[Xi,Xp_i(£)], [Xi,Xk]\ 

= [X i ,Xp- I ^ +ek+e{ \ + [X^Xp-HQ+e^]. 

To conclude the proof, we need to show that the sum 

(3.31) 22 C ^[ X i^ X IB-I(i)+e e +e k ] + C tp[ X ii X P-I(C)+e Lk ] 

is equal to E^ +efci9 ce,p+e k [ x i> x P+e k -i(i)+e e ], i.e., to 

(3.32) ce ) p+ eh [X i ,Xi3 +ek _ I (fl +et ]+ c ij3+e h [Xi, X^ +efc _ J( | )+e J . 
For fixed (3, we introduce the notation 

that is well defined if I(£)k < fik- 

Let us rewrite (I3.3ip . In the second summation in (I3.3ip . we perform the change 
of indices I = i.k. Then we have I(£) = I(£) + e^ and the summation becomes 

E {x„x s+ , m+] 

/Wfe>i 

fef" h + l-Wk (/3 + e fc -/(£))!(/(£) -e fc )! [ 

l<I(I)k<Pk I(Z) k =/3 k +l 



Therefore, the sum in (13.311) is 



em - E »w + E j^Bmf {l) 

fO QO\ l<HQk<Ph 

[6.66) 

+ , F £ (/3 + e fc -/(£))!(/(£) -e fc )! [Xi ' X/3+e - /W+cJ 

I(0 fc =j8 fc +1 



EXTREMAL CURVES IN NILPOTENT LIE GROUPS 13 

We split the first summation in H3.33|) according to whether I(£)k = or not. Notice 
that £ G Ap tq is equivalent to £ G Ap+ ek , q and I(£)k < fa- Then we have 

E = E + E $ w 

/(%=() l<J(*)k</8* 
/(%=0 

and, by (I3.33p . we obtain 

oan= E ATr^k* M+ E •«> + E &t^tm;* w 

I(£) k =0 l<I{t)k<Pk l<I(t)k<Pk 

+ (P + e k -im(I(£)-e k )\ [Xi > X ^~^ ■ 



Hence, the sum in (13.31 j) is 

ft + 1 - !(()„ 



&€.Ap+e k ,q 

0<I(i) k <Pk 

+ £ (/3 + e fc -/(f))!(/(f)-e fc )! [Xi '^- /w+c ' ] 

and it can be easily checked that the right hand side of the previous formula equals 
f)3.32p . This concludes the proof. □ 

Let f S;r denote the free nilpotent Lie algebra over K of step s and rank r. Let 
n = dim(jy jS ) be the dimension of the algebra. Let us fix a Hall basis for f s r . Then 
for any £ G {l,...,n} we have a multi- index I(£) G X, see (13.251) . and moreover 
there is a partial order ^ on indices, see Definition 13.11 Finally, recall the notation 
x a = x" 1 ■ . . . ■ i™" for ieK" and a el. 

The following theorem is proved by M. Grayson and R. Grossman, see [6j Theorem 
2.1]. 

Theorem 3.3. The vector fields X 1 , . . . , X r inM. n 

(3.34) ^HEH^r^W XeRH ' 

with i = 1, . . . , r, generate a Lie algebra isomorphic to f sr . 



The index £ = i is included in the sum in ( I3.34p and gives the summand d/dxi. In 
[6], the authors use a slightly different notation. Moreover, there is no £ ^ 1 such 
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that 1 ^ £ and thus X\ = d/dx\. Finally, notice that Xi, . . . ,X r satisfy assumption 

(EES]). 

By Hall's construction, the generators X\, . . . ,X r can be completed to a basis 
Xi, . . . , X n of the Lie algebra. We call Xi, . . . , X n a Hall-Grayson-Grossman basis of 
vector fields in R n . 

In [5], the authors also describe the elements Xj with j > r. For o6l with q^O, 
define the minimal order of the monomial x a as 

m(x a ) = min |j 6 {1, . . . , n} : aij > 0}. 

For a polynomial P(x) = ^2 h=1 ChX° h with c/, ^ and £ I, ct), ^ 0, we define the 
minimal order m(P) = max/ !=lv ^m(i a ' 1 ). 

For z, £ £ {1, . . . , n} with i ^ £, let us define the monomial 

(3.35) P„(x) = ( J( ^_ J(0)| ^ W ~ JW . * e R?t - 
Notice that (I3.34p can be rewritten as 

d 

Xi{x) = y^P il (x) — , i = l,...,r. 

era* 

Lemma 3.4. (jSJ Lemma 2.3]) //Xj an element of the Hall basis constructed as 
Xi = [Xj,X k ], then 

(3.36) X t {x) = + ^2q u ( x ) — , x G R", 

1 1=1 1 

where Qu are polynomials with no constant terms satisfying m{Qi^) < k and Pa are 
monomials of the form (13.351) satisfying k < m(P#) < i for all t ^ i. 

The exponential mapping of the second type related to an ordered system of vector 
fields Xi, . . . , X n is the mapping ^ : R n — > R™, when globally defined, 

(3.37) *(ar) = e XlXl o . . . o e XnXn (0), x G R n , 

where e tx denotes the flow of the vector field X with parameter t G R. When G is a 
Lie group with group law • and X±, . . . , X n G g = Lie(G), we can equivalently define 
the mapping ^ : R n — > G as 

(3.38) *(ar) = exp(x n X n ) • . . . • exp(xiXi), x G R n , 

where exp : q — > G is the exponential mapping. 

We prove that a Hall-Grayson-Grossman basis of vector fields in R n induces expo- 
nential coordinates of the second type. 

Proposition 3.5. Let X\, . . . ,X n be a Hall-Grayson-Grossman basis of vector fields 
in R n . Then we have x = ^>(x) for all x G R n . 
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Proof. Let ei, . . . , e n be the standard basis of M n . By the structure (I3.36P of X n , we 
deduce that e XnXn (0) = x n e n . Assume by induction that for some 1 < i < n we have 

n 

(3.39) (a) = e XlXl 0...0 e x ^ ( ^ x h e h ^j . 

h=i+l 

Assume first that i > r. Then we have A, = [A^A^.] for some k < j < i, and A, 
is of the form (13.361) . Since m(Qa) < k < i for all £ — 1, . . . , n, the polynomial Qn 
vanishes along the flow of Aj starting from Ylh=i+i x h e h- Because m(Pu) < i for £ ^ % 
with i ^<£, the monomial Pa also vanishes along the same flow. When £ = i, we have 
Pa — 1. It follows that 

n n 

(3.40) e^( Yl Xheh ) = J2 Xheh - 

h=i+l h=i 

This proves the inductive step when i > r. 

Assume now i G {1, . . . , r}. We have Xi = d/dxi and for i — 2, . . . , r the condition 
i £, £ 7^ i, implies that m{Pn) < i. The same argument as above proves (13.401) also 
when i — 1, . . . , r. □ 



4. Integration of the adjoint equations in free nilpotent groups 

Let G be a free nilpotent Lie group of dimension n and rank r. The Lie algebra g 
of G is isomorphic to a Lie algebra of vector fields in M n that are left invariant with 
respect to some product structure. Let Ai, X n be a Hall-Grayson-Grossman basis 
of this Lie algebra. Recall that the generators X\, . . . , X r have the form (I3.34p . In 
this section, we integrate the system of differential equations (I2.17P in MJ 1 with the 
structure constants G M determined by the basis Xi, A n : 

n 

(4.41) [X l ,X J ] = J24X k , i.j 1 //. 

k=i 

Let us also introduce the generalized structure constants c\ a G M for any multi-index 
a 6 I = N" and i,k G {1, . . . , n}. These constants are defined via the relation 

n 

(4-42) [X l ,X a ] = J2ctX k . 

k=l 

Recall the definition ( I3.24p for the iterated commutator [Aj, X a ]. 

For any % = 1, . . . , n and a G X, define the linear mapping <fi ia : W 1 — > R 

Notice that (pio(v) = v^. The following polynomials are central objects in the integra- 
tion formulae for the adjoint equations. 
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Definition 4.1. For each i G {1, ... ,n} and v G M. n , we call the polynomial Pf : 
W 1 -» R 

(4.44) py(x) = Y,<l>ia(v)x a , xeW\ 

extremal polynomial of the free nilpotent group G with respect to the basis X±, . . . , X n 
ofg = Lie(G). 

Remark 4.2. Notice that the generalized structure constants c\ a in f)4.42p satisfy c\ a = 
if d(i) + |a| > s, where s is the step of the Lie algebra. Then the polynomial 
has homogeneous degree at most s — d{i). Recall that, by definition, the homogeneous 
degree of a monomial x a , a G X, is d(a) := YTj=i a j ^0) an d the homogeneous degree 
of a polynomial ^2 i=1 C{X oti , with 7^ and «j G X, is maxi = i j .„ t jv rf(aj). 

Proposition 4.3. Extremal polynomials have the following properties. 

(i) If £,i — 1, . . . ,n are such that i < t and ve 7^ 0, then P£ 7^ 0. 

(ii) If v el" zs stzc/i i/iai = for all i = 1, . . . , r, then v = 0. 

(iii) For all % — 1, . . . , n and t> G lR n we /iawe -P^(O) = Wj. 

Proof. Let us prove (i). As % -< £, we have Xg = [Xi,X a ] for a = I(£) el. It follows 
that cf Q = 5^, and thus, as vt 7^ 0, 

P"(x) = t^v e x a + V ^(v)^^0. 
Of! — ' 

Statement (ii) is an easy consequence of (i), while the proof of (iii) is elementary and 
relies upon the identity cf = 5ik- □ 

Remark 4.4. We point out the following implication: if r 2 := dimg 2 and v G K n is 
such that P? = for all i = r + 1, . . . , r + r 2 , then v r+ i = v r+ 2 = ■ ■ ■ = v n = 0. 

Indeed, arguing by contradiction we assume that there exists j with d(j) > 2 such 
that Vj 7^ 0. By Proposition 14.31 (iii) we have 

u r+1 = Vt+2 = ... = Vr+r2 = 0, 
hence d(j) > 3 and we can write 

for a suitable h > 2. The element [Xj , X^] belongs to the Hall basis and, in particu- 
lar, [XjQjXjj] = X k for a suitable k of degree 2. Since k -< j, Proposition 14.31 (i) gives 
P% 7^ 0, a contradiction. 

The following lemma shows that the linear subspace generated by extremal poly- 
nomials is closed under derivatives along left invariant vector fields, in a way related 
to the structure of q. 
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Proposition 4.5. For any v G K n and z,j G {1, . . . , n} there holds 

n 

(4.45) X t P? = J24 P k- 

k=l 

In particular, XiP? = for any % — 1, . . . , n and v G W 1 . 

Proof. Let !)6K" and j G {1, . . . , n} be fixed. We argue by induction on the degree 
of i. If d(i) = 1, i.e., i = 1, . . . , r, we have by f)3.34p 

fe=i oex v 7 

We now use the following polynomial identity 



_i \\a\ + \I(£)\ n / i \ |/3| 



This identity will be proven later, in a crucial step of the proof of Theorem 14.61 see 
equation ()4.53|) . From (I4.46P we obtain 

w? = -E"*EE 44. L ir-*' = E 4^- 

This ends the proof of the induction base. 

If d(i) > 2, we have X$ = [X P ,X U ] for some p, u with d(p),d(u) < d(q). By 
inductive assumption, we have 

y r> v y v r> v v y ~p v 



n n 
k=l k=l 

n 

E( r .k Ji pv _ k h pv\ 
V^uj^pk 1 h u pj u uk 1 h) 



h,k=l 



^ ] ( C ju C pk C pj C uk)-^'h- 
h,k=l 

The Jacobi identity [[X i; X u ],X p ] + [[X^X^Xj] + [[X p ,Xj],X u ] = yields 

n 

E k h _|_ k h , fc h _ n 
^ju^kp ' ^up^kj ' ^pj^ku u - 

k=l 

Using this identity and c^ p Cj k = Cp U c kj = 8 ik c^, we obtain 

n n 
XiP j = zJ C up C, jk P h = z2l C % P h- 



h,k=l h=l 
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This completes the proof. □ 

We identify a free nilpotent Lie group G with W 1 via exponential coordinates of 
the second type related to a Hall-Grayson-Grossman basis X\, . . . ,X n , as explained 
in Proposition 13.51 

Theorem 4.6. Let G = W 1 be a free nilpotent Lie group, let 7 : [0, 1] — > G be a 
horizontal curve such that 7(0) = 0, and let A : [0, 1] — > lR n be a Lipschitz curve. The 
following statements are equivalent: 

A) The curve A solves the system of equations (I2.17P . 

B) There exists !)6K" such that, for all i = 1, . . . , n, we have 

(4.47) A 4 (t)=Xf( 7 (t)), te[0,l], 
and in fact v = A(0). 

Proof. We show that the curve A defined by (14.471) solves the system (I2.17P with 
initial condition A(0) = v. By the uniqueness of the solution, this will prove the 
equivalence of the statements A) and B). Notice that the curve A defined via (I4.47P 
satisfies A(0) = v by Proposition 14.31 part (iii). 

We preliminarily compute the derivative of t 1— > 7(t) Q for any multi- index a G X. 
We have 

,n n 

(4.48) - 7 a = Yl II = E ^ a -^it- 

1=1 p^te 1=1 

Since 7 is horizontal we have 

r 

(4.49) 7 = ^ 79 X 9 ( 7 ) a.e. on [0,1], 

9=1 

where X\, . . . , X n is the fixed Hall basis and r > 2 is the rank of the group. 

For any I = 1, . . . , n we can compute the derivative 7^ starting from (14.491) and 
from the formula (I3.34p for the generators Xi, . . . ,X r of the Lie algebra. We have 
to consider the index £ = 1, . . . ,r and the multi-index I(£) e X such that Xg = 
[Xi , X see (I3.26p . and then look at the £-th coordinate of the vector field Xg . 
We then find 

(4.50) % = 7^ 1 J( ; £) , I m a.e. on [0, 1]. 
By 007]), (g3SD , ( ESSD and the definition of P?, we obtain 



which is equivalent to 



m » a\I(£)\ 7 ' 

a£l 3=1 1=1 v ' 



q=l j = l a€l£:l =q V >' 
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On the other hand, the right hand side of (I2.17P is 

r n r n n , s |«| 

(4.52) - £ % £ c?„A t = - E %E 1 E E "Wo^grV- 

g=i fe=i g=i i=i fc=i ^ex ^ 

Comparing the lines (I4.5ip and ()4.52|) . we see that it is sufficient to show that for any 
fixed i,j — l,...,n and q — 1, . . . ,r the following identity of polynomials is verified 

(4.53) e E <£«< L ^ m -^ m = - E E <W=$-**. 

aeXti =q ■ \ )■ k=l P&X H ' 

On the other hand, this identity is equivalent to the following combinatorial identity, 
for any fixed (3 e X: 

» (-1)1^1 

^ ^ «!/(£)! ~ Z^Vfcfl f3\ ■ 

e-.e =q v y k=i r 

a:a-e l +l[e)=P 

Since X 1; . . . , X n is a basis, this is in turn equivalent to 

E( E ^^-W 1 !^ 

j = l M =<7 fc=l i=l 

fc=i p " 



/3! 

We may rearrange the last identity in the following way: 

(4-54) Yl ^-^—^[X t ,X a ] = - y —-l-[[X t ,X q ],X^ 

ti = q a.i\t). p. 

a:a-et+I(i)=P 

Notice that the condition a — e^ + 1(1) = implies I(£) < (3, because I(£)e = 0. In 
particular, the left hand side of (I4.54p is 

(_1)M+|/MI (_i)l/?l 
2^ <*' aim ^ X J = - 2^ ((3 - I(£))\I(£)\ [X " X ?-m + e e ] ■ 

a:a-e e +I(£)=/3 !{£)<P 



We conclude that ( I4.54p is equivalent to the identity 

^ {(3 -I{£))U{£)\ [Xi'Xp-W+eil = J^\ x ^ X qI x ^ 



which is precisely identity (I3.29P in Lemma 13.21 This concludes the proof of the 
theorem. □ 
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Remark 4.7. Theorem 14.61 provides an explicit integration of the second equation in 
the Hamiltonian system (12. 9p . The first Hamilton's equation in (12. 9p . namely the 
equation 7 = ^(7, £), reads 

r 

7 = -^(£,^( 7 )>^-(7). 

By (I2TT5D and (T4T47p . we have 

(e,X,( 7 )) = A,(7W) = ^(7W) 

with f = A(0). By the formula (I3.34j) for Xj, we obtain the following system of 
ordinary differential equations for normal extremals 7 : [0, 1] — > G = M. n 

1 IU)\ 1 jKV dx, 

j=i j<t ^ ' 

In order to characterize abnormal curves, we introduce the following algebraic va- 
rieties. 

Definition 4.8. Let G = M. n be a free nilpotent Lie group of rank r. For any v G R n , 
v 0, we call the set 

Z v = {x G R n : P?(x) = ...= P?(x) = 0} 

an abnormal variety of G of corank 1 . 

For linearly independent vectors v±, . . . , v m G M n , m > 2, we call the set Z m PI ... R 
Z Vm an abnormal variety of G of corank m. 

Abnormal varieties depend on the system of coordinates of the second type induced 
on G by a Hall-Grayson-Grossman basis Xi, . . . , X n for the Lie algebra of G. 

Theorem 4.9. Let G = M. n be a free nilpotent Lie group and let 7 : [0, 1] — > G be a 
horizontal curve with 7(0) =0. The following statements are equivalent: 

A) The curve 7 is an abnormal extremal of corank m > 1 . 

B) There exists m linearly independent vectors vi, . . . ,v m el" such that 7(£) G 

z Vl n...nz Vm for alite [0,1]. 

Proof. Recall that the property of having corank m for 7 is equivalent to the existence 
of m linearly independent solutions to the system (I2.17p . 

Let 7 be an abnormal extremal and let A : [0, 1] — > W 1 be a Lipschitz curve solving 
the system (12.171) and such that A 7^ pointwise. By Theorem 14. 6[ there is v G M n , 
ti / 0, such that A, = ^"(7) for all i = 1, . . . ,r. For abnormal extremals we have 
Ai = . . . = A r = on [0, 1], i.e., j(t) G Z v for all t G [0, 1]. This shows that B) follows 
from A). 

On the other hand, if B) holds then the curve A defined in (I4.47P for any v = 
Vi,...,v m satisfies the system (I2.17P by Theorem 14.61 and moreover Ai = ... = 
A r = 0. From v ^ it follows that A(0) = v 7^ 0, and from the uniqueness of the 
solution to (I2.17P with initial condition it follows that A 7^ pointwise on [0,1]. If 



EXTREMAL CURVES IN NILPOTENT LIE GROUPS 



21 



Vi, . . . , v m are linearly independent, then the corresponding curves A are also linearly 
independent. □ 

Remark 4.10. Notice that when v ^ the zero set Z v is nontrivial, i.e., Z v ^ M. n , 
because by Proposition 14. 31 (ii) there is at least one index i = 1, . . . ,r such that P? ^ 
is not the zero polynomial. 



The following corollary easily follows from Definition 12.51 

Corollary 4.11. Let G = ~R n be a free nilpotent Lie group with stratified algebra 
= 9i © 02 © " • " © 0s and let 77 = dim(0x) and r 2 = dim(g 2 ). Let 7 : [0, 1] — > G be a 
horizontal curve such that 7(0) = 0. The following statements are equivalent: 

A) The curve 7 is a Goh extremal. 

B) There exists v G R n , v ^ 0, such that for all i = 1, . . . ,77 + r 2 and for all 
t G [0, 1] there holds 

(4.55) P*( 7 (t)) = 0. 

Remark 4.12. Given v G lR n with v 7^ 0, we call the set 

G v = {xeR n : P*{x) = for alH = 1, . . . , r x + r 2 } 

a Goh variety of the free nilpotent group G. By Proposition 14.31 (iii) we have v j = 
for alH = 1, . . . , 77 + r 2 . Therefore, by Proposition 14.31 (ii) and Remark 14.41 there are 
at least one index i G {1, . . . , 77} and one index j G {77 + 1, . . . , 77 + r 2 } such that 
and P/ ^ 0. 

The polynomials P? with z = 77 + 1, . . . , 77 + r 2 completely describe the horizontal 
sections of G v . Namely, let 7 : [0, 1] — > R™ be a horizontal curve such that 7(0) = 
and 

7(t) e{i6l": P?(x) = for alH = 77 + 1, . . . , 77 + r 2 } 

for all t G [0, 1], where v G M. n is a vector such that v , = 0, for alH = 1, . . . , 77. Then, 
by Proposition 14.51 for any index i — 1, . . . , 77 and for any t G [0, 1] we have 

, ri ri n 

fc=i fc=i £=1 

because the only indices £ involved in the previous sum are those with degree 2. We 
deduce that P i 1 '(7(t)) = ^(7(0)) = P^O) = v> = 0. 



5. Extremal curves in stratified groups 

In this section, we give a partial classification of extremal curves in general stratified 
Lie groups. The description is not optimal because it involves a "lifting" procedure. A 
more complete result, e.g., a purely algebraic characterization of abnormal extremals, 
seems to require an explicit extension of Grayson- Grossman results to nonfree nilpo- 
tent groups. 
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Let Yi,...,Y r , r > 2, be smooth vector fields in R n and let ir : R n — y R m , with 
m < n, be a smooth mapping. Let us consider the following system of vector fields 
in R m : 

(5.56) Xi = 7r,(yi),...,X r = 7r*(y r ), 

where 7T* is the differential of 7T. Here and hereafter we assume that Xi, . . . ,X r are 
linearly independent at each point. 

Let 7 : [0, 1] -> R m be a horizontal curve for X 1; . . . , X r with 7(0) = G M™. Then 
there are controls hi, . . . , h r G L 2 ([0, 1]) such that 

r 

(5.57) j(t) =^2h j (t)X j (j(t)), for a.e. t G [0,1]. 

i=i 

We call the curve « : [0, 1] -»■ R™ such that w(0) = and k(t) = YJ j= i hj(t)Yj(K(t)) 
for a.e. t G [0, 1] the lift 0/7 to IR™. 

In the sequel, we denote by T*R n and T*]R m the cotangent spaces to M n and M' m , 
respectively, and we denote by ir* : T*W m — > T*W n the pull-back mapping induced by 
7r. For the sake of completeness, we provide the proof of the following easy fact. 

Proposition 5.1. Let 7 : [0,1] — > lR m be a horizontal curve for Xi, . . . , X r with 
7 (0) = and let k : [0, 1] -»■ R m be the lift 0/7 to M n . J/^ : [0, 1] ->■ T*W m is a dual 
curve for 7 then tt*^ : [0, 1] — > T*M. n is a dual curve for k. 

Proof. Let hi, . . . ,h r G L 2 ([0, 1]) be the controls of 7 as in (I5.57P and let $ : [0, 1] x 
]R m — > IR m be the flow in IR m associated with these controls. Namely, let <&(t,x) = 
7a; (t) where 7^ : [0, 1] — > M m is the solution to the problem 

r 

ix = Yl h i X i^x) a - e - and 7*(0) = x G R m . 

We also let $ t (x) = $(t, ar). Finally, let us denote by * : [0, 1] x ]R n -> ]R n the flow in 
W 1 associated with the same controls. 

By the characterization (12.71) of dual curves, we have 

(5.58) at) = (^rm, 

for some ^(0) G T *M m , ^(0) ^ 0. Above, (^^^(O) is the pull-back via the diffeo- 
morphism ^^T 1 : R m — >■ R m . 

Let 7] = tt*C, be the pull-back to R n of ^ by ir. We claim that for any t G [0, 1] we 
have 

(5.59) rj(t) = ^T'Yvio), 

and thus rj : [0, 1] — > T*R" is a dual curve for the lift k to R n of 7. 

To prove this claim, we preliminarily show that 7r commutes with the flows $ and 
Namely, for any y G R™ and t G [0, 1] we have: 

(5.60) 7r(tf t (j/)) = $ t (7r(y)). 
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This identity holds when t = because 7r(^ (y)) = n(y) = & (7i(y)). Let us compute 
the derivatives in t of the left and of the right hand sides in (I5.60p . We have: 

4*(*i(v))=*.(4**(v)) = ^x>(^(^))) 

3=1 

r r 

= £M*k.(W«(v))) = J2 h ^) x MMy)))- 

3=1 3=1 

In the last line, we used (I5.56p . On the other hand, we have 

3=1 

It follows that the curves t \— > 7r(^/ t (y)) and t i— > ^ t (ii(y)) solve the same differen- 
tial equation with the same initial condition. Now, identity (I5.60p follows from the 
uniqueness of the solution. The same argument proves that 

(5.61) 7T O = $-1 o 7T, te[0,l]- 

Now we prove our main claim (15.591) . By (12. 7ft . by the composition rule for the 
pull-back, and by (15.611) . we have 

= ($- 1 o 7 r)*e(0) = (7r o*- 1 )*e(0) 

= (^ l )*n*(aO)) = (*?M0). 
This ends the proof. □ 

Remark 5.2. The statement of Proposition 15 . 1 1 can be improved in the following sense: 
if 7 is an abnormal (respectively, Goh) extremal with dual curve £, then k is an 
abnormal (resp., Goh) extremal with dual curve it*!;. 

Remark 5.3. Let g be a quadratic form on the distribution T> on IR m spanned by 
Xi, . . . , X r , and let h = n*g be the pull-back of g to the distribution £ on M™ spanned 
by Y\, . . . , Y r . Then, tt preserves the length of horizontal curves. Hence, if 7 is a length 
minimizing curve in (IR m , T>,g), then the lift k of 7 to R n is a length minimizing curve 
in (R n ,£,h). 

Now we pass to stratified Lie groups. Let G be a stratified m-dimensional Lie group 
of step s > 2 and rank r > 2. Its Lie algebra q admits a stratification 

= gi ... ®Q S , 

where = [01, &] for i = 1, . . . , s — 1 and gj = {0} for i > s. The first layer jjx has 
dimension r. 

Let F be the free nilpotent Lie group of step s > 2 and rank r > 2. Let f be its Lie 
algebra and let n = dim(f). Let Y 1} . . . ,Y r be generators of f. There is a surjective 
Lie group homomorphism 7r : F — > G, with differential mapping 7T* : f — > Q, such that 
Tr^Yi G 0i, i = 1, . . . , r. We complete Yi, . . . , Y r to a Hall basis K 1; . . . , Y n of f. By 
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exponential coordinates of the second type, we can identify F with M n and assume 
that Yi, . . . ,Y n is a Hall-Grayson-Grossman basis of left invariant vector fields on 
R n = F. 

Let S C {1, 2, . . . , n} be a set such that the vector fields 7T*Y S with s 6 5 form a 
basis for g. We have m = #S and S = {si < s 2 < . . . < s m }. We relabel the basis in 
the following way 

Xi = ir*Y s ., i = l,...,m. 

As q has rank r, then the vector fields Xi = 7r*Yi £ jji, % — 1, . . . , r, are generators of 
0. There are constants Qj G K such that 

rn 

(5.62) ^c,,,V ; , 

In the following theorem, G = lR m is a stratified Lie group with rank r and step s, 
and F = R n is the free nilpotent Lie group with rank r and step s. The lift procedure 
from G to F is defined with respect to the systems of vector fields fixed above. 

Theorem 5.4. Let 7 : [0, 1] — > G be a horizontal curve with 7(0) = and let 
k : [0, 1] — > F be the lift of 7 to F. If 7 is an extremal curve with dual curve 
X : [0, 1] -»■ E m then there is v el" such that 



3lj V Sj , 



(5.63) = 

and £/ie coordinates of A satisfy, for all i = 1, . . . , m, 

A,(0 = ^(«(0), ie[0,l]- 

Proof. If 7 is an extremal curve in G with dual curve A, then k is an extremal curve 
in F with dual curve \i = tt*X, by Proposition 15.11 By Theorem 14. 6[ there is v e R™ 
such that the coordinates of \x in the dual basis of Yi, . . . , Y n are 

^(t) = if («(*)), tG[0,l], 

for any z = 1, . . . , n, and in fact we have v = //(0) = 7r*A(0). By (15.621) . we have for 
i — 1, . . . , n 

m m 

Hi = ix(Yi) = 7r*A(r,) = X(^Yi) = J2(aK x i) = 

3=1 j=l 

At t = 0, this identity implies the relation (I5.63P for v. On the other hand, for any 
i = 1 , . . . , m we have 

A, = AM = A(7r*y si ) = //(nj = ^ = ^(*)- 

□ 

6. Examples and applications 

In this section, we discuss some examples on how theorems and formulae of Sections 
H] and [5] can be applied. 
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6.1. Regularity of geodesies in stratified groups of step 3. We give a short 
and alternative proof of a result proved in [IB"] . 

Theorem 6.1. Let G be a stratified Lie group of step 3 with a smooth left invariant 
quadratic form g on the horizontal distribution T>. Any length minimizing curve in 
(G,V,g) is of class C°° . 

Proof. By Proposition 15.11 and Remark I5.3[ we can assume that G is free. Let n and 
r be the dimension and the rank of G, respectively. By contradiction, assume there is 
a length minimizing curve 7 : [0, 1] — > G that is not of class C°°. Then 7 is a strictly 
abnormal extremal (normal extremals are of class C°°) and thus a Goh extremal. 

We can assume that 7(0) = 0. By Theorem I4.9[ there are an index i = 1, . . . , n 
with d(i) = 2 and v G M. n , v ^ 0, such that Pi(j) = 0. Notice that the polynomial 

£:d(£)=l 

has homogeneous degree 1. On differentiating in t the identity ^(7) = 0, we obtain 
(6.64) £ c>, 7 £ = 0. 

£:d(£)=l 

Let f be the (free) subalgebra of g = $ji © $j 2 © $j 3 = Lie(G) generated by the 
subspace fi C 0i 

fi = I ^2 x iXe e 01 : £ 4 v J ar / = °}» 

£:d(£)=l j:d(j')=3 
&d(^)=l 

where X 1; ...,X r G 0i are Hall- Grayson- Grossman generators. As Pi(x) 7^ 0, we 
have dim(fi) = r — 1. Let FcGbe the stratified Lie group with Lie algebra f. By 
f)6.64p . 7 is in f and hence the curve 7 is in F. Moreover, 7 is a length minimizer in 
F for the restricted quadratic form, and thus 7 is a Goh extremal in F (because it is 
not smooth). 

We can repeat the above reduction argument to conclude that 7 is a Goh extremal 
in a free nilpotent group of rank 2. Now the equation 

m-r) = - E 4w=o, 

te{i,2} 

where is a nonzero polynomial, implies that 7 is a line, and thus a smooth curve. 
This is a contradiction. □ 

6.2. Regularity of generic length minimizing curves in stratified Lie groups 
of rank 2. We prove that in stratified Lie groups of rank 2 length minimizing curves 
are "generically" smooth. This is a special case of a deeper series of results by W. 
Liu and H. J. Sussmann on regular abnormal extremals, see |12j . 
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Let G be a stratified Lie group with Lie algebra g = Qi © • • • © q s of rank 2 and 
step s > 3. We have r% = dim(gi) = 2, r 2 = dim(g 2 ) = 1, and r 3 = dim(g 3 ) G {1, 2}. 
We fix a Hall- Grayson- Grossman basis X 1; . . . , X n , and we identify G with R n . For 
a dual curve A : [0, 1] — > R™, we let A^ = A 4 when r 3 = 1, and A® = (A 4 , A 5 ) when 
r 3 = 2. 

Proposition 6.2. Let 7 : [0, 1] — > G = R™ be a Goh extremal with dual curve 
A : [0,1] ->• R™. Assume that \<®(t) ^ for all t G [0,1]. TTien 7 zs an analytic 
curve. 

Proof. By Proposition 15. II and Remark 15.21 we can assume that G is free; in particular, 
dim(g 3 ) = 2. By Corollary 14. 11[ there exists a v G R™, f 7^ 0, such that for all t G [0, 1] 
we have 

(6.65) 7$ G S = {x G R n : P^x) = 0}, 

where -P 3 (x) 7^ is a nonzero polynomial of the form (14.441) . 
By Proposition 14.51 we have 

XiP^x) = -PJftx) and X 2 P%{x) = -P^x). 

By Theorem 14.61 A4 = -P4 (7) and A5 = P5 (7), and the assumption A® 7^ on [0, 1] 
implies that 

(^3 (7)) 2 + (X 2 P2 (7)) 2 7^ 0. 
In particular, VP 3 7^ and thus E is an analytic hypersurface of R n in a neighbor- 
hood of the support of 7. Moreover, the distribution V(x) = span{Xi(a;), X 2 (x)} is 
transversal to T^E: 

dim^E n V(x)) = 1, for x G 7([0, 1]). 
Inasmuch as 7 is horizontal and by (I6.65p . this implies that 7 is analytic. □ 

Remark 6.3. By Theorem 5 in [121 P- 59], Goh extremals as in Proposition 16.21 are 
locally length minimizing. 

6.3. A strictly abnormal curve. We review Gole-Karidi's example [5] of a strictly 
abnormal curve in a stratified group. 

Let G be a (free) nilpotent Lie group of rank 2 and step s > 4, as in the previous 
example. We fix a Hall basis Xi, . . . , X n and we highlight the first commutators 

(6.66) X 3 = [X 2 ,X 1 ], X 4 =[X 3 ,X 1 ], X 5 = [X 3 ,X 2 ], X 6 = [[X 3 ,X 1 ],X 1 ]. 

We look for a Goh extremal 7 : [0, 1] — > G = R n . To this aim, let us consider the 
polynomial 

a£l a£l ' j=l 

We look for a v G R n , v 7^ 0, and for a curve 7 such that -P 3 (7) = 0. The identities 
-^i (7) = P2 (7) = will then follow from Proposition 14. 5[ by the argument of Remark 
I4.12[ provided that we can choose v\ = v 2 = 0. 
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The coefficient of V5 in the polynomial P3 {x) is 

because the unique multi-index a G X such that c| a 7^ is a = (0, 1, 0, . . . , 0) and for 
this a we have c\ a = 1, see f)6.66p . 

The coefficient of vq in the polynomial P$(x) is 

EC— "HH 1 

because the unique multi-index a G X such that Cg a 7^ is a = (2, 0, . . . , 0) and for 
this a we have c\ a = 1, see again (I6.66p . 

With v 5 = vq = 1 and with Vj = otherwise, the polynomial P£{x) is 

= is? -a*. 

Now we clearly see that the horizontal curve 7 : R — >■ M n such that 71 (i) = t and 
72 (^) — \ ft is a Goh extremal, by Corollary 14 . 1 1 1 and Proposition 14.51 

We prove that 7 is a strictly abnormal curve. If 7 were a normal extremal with 
dual curve A, then by (I2.8p and (12.1 5p we would have 

7j = hj = -(£, Xj) = Xj, j = 1, 2. 
This yields Ai = 1 and A2 = t. Moreover, the system of equations (I2.17P is 

n 

Ai = -X)Wi + 4*} A *> i = l,-..,n. 

k=l 

When i = 1 we obtain = 0, whereas for % = 2 we obtain A3 = —1. This is a 
contradiction. 

Remark 6.4. As proved in [5], the curve 7 above is locally length minimizing. Indeed, 
with the previous choice of v, we have by Proposition 14.51 that 

Therefore, 7 satisfies the assumptions in Proposition 16.21 and is locally length mini- 
mizing by [12] . 

Let us give here the complete formula for the polynomial P3 (x) when G is a free 
nilpotent Lie group of rank 2 and step s — 6. This group is diffeomorphic to 1R 23 . 
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The polynomial is 

9 QOO 9 O 

P (Xj = U 3 - - V 5 £ 2 + V 6 — + V 7 X 1 X 2 + V 8 — ~ Vg— - V W — V n — V 12 — 

2 2 2 2 o 

+ ^13(^4 + X X X 3 ) + v 14 (x 5 + x 2 x 3 ) 

4 3 2 2 2 4 

-x> 1 x 1 v 12 x 1 3 **^2 

+ ^ + «„— + ^1 7 — + Vi 8 — + V 19 - 

2 2 
+ ^2o(^6 ) + «21 (^7 - ^1^2X3) + ^22(^8 Y~ ) + ^23(^2^4 - XiX B ). 

The homogeneous degree of the polynomial is at most 4. The variables xg, . . . , X23 do 
not appear. 

6.4. Rank 3 and step 4. Let G be the free nilpotent Lie group of rank 3 and step 
4. This group is diffeomorphic to IR 32 . By Corollary 14.111 and Remark I4.12[ Goh 
extremals of G starting from are precisely the horizontal curves 7 contained in the 
algebraic set 

S = {x G M 32 : Pl{x) = P»(x) = Pg(x) = 0}, 

for some v G M 32 such that V\ — . . . — v% — 0. We list the the polynomials defining 
this algebraic set: 

Pl{x) = -XxV 7 - X 2 V 8 - X 3 Vg + X 5 V 30 + X 6 V 31 

2 22 

X> 'JC 00 

+ y^l5 + XiX 2 Wl6 + ^1^3^17 + y^l8 + X 2 X 3 V 19 + yH 2 

P£(x) = -XiV 10 - x 2 v u - x 3 v 12 - X A V 30 + XqV 32 

x^ 1 x^ 1 x^ 1 

+ y v 2 i + x x x 2 v 22 + X X X 3 V 23 + y v 24 + x 2 x 3 v 2 5 + y v 26 

Pg (x) = ari(v B - t>n) - x 2 v 13 - x 3 v M - x A v 3l - x 5 v 32 + xl(--v l7 + -v 22 + v 30 ) 

x^ 1 

+ XiX 2 (-Vig + V 24 + V 31 ) + X!X 3 (-V 2 o + V 25 ) + y^29- 

The set E is an intersection of quadrics. 

When v 7 = 1 v ls = 2 and Vj = otherwise, we have P%(x) = x\ — Xi, P£(x) = 
Pq(x) = 0. Let <p : [0, 1] — > R be any Lipschitz function with 0(0) = 0. The horizontal 
curve 7 : [0, 1] -»■ M 32 such that 7(0) = 0, jx(t) = t 2 , j 2 (t) = t and 73 (t) = <p(t) is a 
Goh extremal with purely Lipschitz regularity. 
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